{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We want to build a CPU dataset.\n",
    "There is pointer on https://github.com/felixsteinke/cpu-spec-dataset\n",
    "\n",
    "## AMD Dataset\n",
    "\n",
    "Manually Export Data as CSV/Excel from https://www.amd.com/en/products/specifications/processors\n",
    "Website Screenshot\n",
    "\n",
    "blob:https://www.amd.com/fc70f2d3-27d4-4bf1-a466-9912927167de\n",
    "\n",
    "## Ampere Dataset\n",
    "\n",
    "Extracted from https://amperecomputing.com/briefs/ampere-altra-family-product-brief with the responsible Web Scraper.\n",
    "\n",
    "cd ampere-web-scraper\n",
    "java -jar ./target/exectuable.jar\n",
    "\n",
    "## Intel Dataset\n",
    "\n",
    "Extracted from https://ark.intel.com/content/www/us/en/ark.html with the responsible Web Scraper.\n",
    "\n",
    "cd intel-web-scraper\n",
    "java -jar ./target/exectuable.jar\n",
    "\n",
    "## CPU Benchmark Dataset\n",
    "\n",
    "Extracted from https://www.cpubenchmark.net/cpu_list.php with the responsible Web Scraper.\n",
    "\n",
    "cd benchmark-web-scraper\n",
    "java -jar ./target/exectuable.jar\n",
    "\n",
    "## CPU World Dataset\n",
    "\n",
    "Extracted from https://www.cpu-world.com/CPUs/CPU.html with the responsible Web Scraper.\n",
    "\n",
    "cd cpuworld-web-scraper\n",
    "mvn -jar ./target/exectuable.jar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extraction from https://www.intel.com/content/www/us/en/ark/products/series/236644/5th-gen-intel-xeon-scalable-processors.html\n",
    "```html\n",
    "<table id=\"product-table\" class=\"table table-sorter sorting tablesorter tablesorter-default\">\n",
    "                                <thead>\n",
    "                                    <tr class=\"tablesorter-headerRow\">\n",
    "                                        <th class=\"product-table-sortable-col product-name-col ark-product-name xlarge-column component header tablesorter-header\" data-component=\"wa_skip_track\" data-component-id=\"1\" data-col=\"productName\" data-sort-field=\"productName\" role=\"columnheader\" aria-sort=\"\" aria-rowindex=\"1\" data-column=\"0\"><div class=\"tablesorter-header-inner\">\n",
    "                                            <div class=\"header-additional-padding\">\n",
    "                                                <div class=\"header-text-space\">\n",
    "                                                    Product Name\n",
    "                                                </div>\n",
    "                                            </div>\n",
    "                                        </div></th>\n",
    "                                        \n",
    "                                            \n",
    "                                                \n",
    "                                                    <th class=\"product-table-sortable-col component medium-column descending tablesorter-header\" data-component=\"wa_skip_track\" data-component-id=\"1\" data-col=\"BornOnDate\" role=\"columnheader\" aria-sort=\"\" aria-rowindex=\"1\" data-column=\"1\"><div class=\"tablesorter-header-inner\">\n",
    "                                                        <div class=\"header-additional-padding\">\n",
    "                                                            <div class=\"header-text-space\">\n",
    "                                                                Launch Date\n",
    "                                                            </div>\n",
    "                                                        </div>\n",
    "                                                    </div></th>\n",
    "                                                \n",
    "                                            \n",
    "                                                \n",
    "                                                    <th class=\"product-table-sortable-col component medium-column tablesorter-header\" data-component=\"wa_skip_track\" data-component-id=\"1\" data-col=\"CoreCount\" role=\"columnheader\" aria-sort=\"\" aria-rowindex=\"1\" data-column=\"2\"><div class=\"tablesorter-header-inner\">\n",
    "                                                        <div class=\"header-additional-padding\">\n",
    "                                                            <div class=\"header-text-space\">\n",
    "                                                                Total Cores\n",
    "                                                            </div>\n",
    "                                                        </div>\n",
    "                                                    </div></th>\n",
    "                                                \n",
    "                                            \n",
    "                                                \n",
    "                                                    <th class=\"product-table-sortable-col component medium-column tablesorter-header\" data-component=\"wa_skip_track\" data-component-id=\"1\" data-col=\"ClockSpeedMax\" role=\"columnheader\" aria-sort=\"\" aria-rowindex=\"1\" data-column=\"3\"><div class=\"tablesorter-header-inner\">\n",
    "                                                        <div class=\"header-additional-padding\">\n",
    "                                                            <div class=\"header-text-space\">\n",
    "                                                                Max Turbo Frequency\n",
    "                                                            </div>\n",
    "                                                        </div>\n",
    "                                                    </div></th>\n",
    "                                                \n",
    "                                            \n",
    "                                                \n",
    "                                                    <th class=\"product-table-sortable-col component medium-column tablesorter-header\" data-component=\"wa_skip_track\" data-component-id=\"1\" data-col=\"ClockSpeed\" role=\"columnheader\" aria-sort=\"\" aria-rowindex=\"1\" data-column=\"4\"><div class=\"tablesorter-header-inner\">\n",
    "                                                        <div class=\"header-additional-padding\">\n",
    "                                                            <div class=\"header-text-space\">\n",
    "                                                                Processor Base Frequency\n",
    "                                                            </div>\n",
    "                                                        </div>\n",
    "                                                    </div></th>\n",
    "                                                \n",
    "                                            \n",
    "                                                \n",
    "                                                    <th class=\"product-table-sortable-col component medium-column tablesorter-header\" data-component=\"wa_skip_track\" data-component-id=\"1\" data-col=\"Cache\" role=\"columnheader\" aria-sort=\"\" aria-rowindex=\"1\" data-column=\"5\"><div class=\"tablesorter-header-inner\">\n",
    "                                                        <div class=\"header-additional-padding\">\n",
    "                                                            <div class=\"header-text-space\">\n",
    "                                                                Cache\n",
    "                                                            </div>\n",
    "                                                        </div>\n",
    "                                                    </div></th>\n",
    "                                                \n",
    "                                            \n",
    "                                                \n",
    "                                                    <th class=\"product-table-sortable-col component medium-column tablesorter-header\" data-component=\"wa_skip_track\" data-component-id=\"1\" data-col=\"MaxTDP\" role=\"columnheader\" aria-sort=\"\" aria-rowindex=\"1\" data-column=\"6\"><div class=\"tablesorter-header-inner\">\n",
    "                                                        <div class=\"header-additional-padding\">\n",
    "                                                            <div class=\"header-text-space\">\n",
    "                                                                TDP\n",
    "                                                            </div>\n",
    "                                                        </div>\n",
    "                                                    </div></th>\n",
    "                                                \n",
    "                                            \n",
    "                                        \n",
    "                                    </tr>\n",
    "                                </thead>\n",
    "                                <tbody>\n",
    "                                    \n",
    "                                        <tr class=\"blank-table-row   seg-server  seg-embedded  seg-retailsku\" data-codename=\"Sapphire Rapids\" data-filter=\"Server\" data-product-id=\"236639\">\n",
    "\n",
    "                                            \n",
    "                                            \n",
    "\n",
    "                                            <td class=\"ark-product-name ark-accessible-color component\" data-value=\"1\" data-component=\"arkproductlink\" data-component-id=\"1\">\n",
    "                                                <div class=\"add-compare-wrap\">\n",
    "                                                    <label class=\"containerCB component\" data-component=\"wa_skip_track\" data-component-id=\"1\">\n",
    "                                                        <input class=\"compare-checkbox compare-toggle\" data-component=\"ark-component\" data-product-id=\"236639\" type=\"checkbox\">\n",
    "                                                        <span class=\"checkmark\"></span>\n",
    "                                                    </label>\n",
    "                                                    <a href=\"/content/www/us/en/products/sku/236639/intel-xeon-bronze-3508u-processor-22-5m-cache-2-10-ghz/specifications.html\">Intel® Xeon® Bronze 3508U Processor</a>\n",
    "                                                </div>\n",
    "                                            </td>\n",
    "\n",
    "                                            \n",
    "                                                \n",
    "                                                    <td class=\"\" data-value=\"\">\n",
    "                                                        \n",
    "                                                            \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                    Q4'23\n",
    "                                                                \n",
    "                                                            \n",
    "                                                        \n",
    "                                                    </td>\n",
    "                                                \n",
    "                                            \n",
    "                                                \n",
    "                                                    <td class=\"\" data-value=\"8\">\n",
    "                                                        \n",
    "                                                            \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                    8\n",
    "                                                                \n",
    "                                                            \n",
    "                                                        \n",
    "                                                    </td>\n",
    "                                                \n",
    "                                            \n",
    "                                                \n",
    "                                                    <td class=\"\" data-value=\"2200\">\n",
    "                                                        \n",
    "                                                            \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                    2.2 GHz\n",
    "                                                                \n",
    "                                                            \n",
    "                                                        \n",
    "                                                    </td>\n",
    "                                                \n",
    "                                            \n",
    "                                                \n",
    "                                                    <td class=\"\" data-value=\"2100\">\n",
    "                                                        \n",
    "                                                            \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                    2.10 GHz\n",
    "                                                                \n",
    "                                                            \n",
    "                                                        \n",
    "                                                    </td>\n",
    "                                                \n",
    "                                            \n",
    "                                                \n",
    "                                                    <td class=\"\" data-value=\"23040\">\n",
    "                                                        \n",
    "                                                            \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                    22.5 MB\n",
    "                                                                \n",
    "                                                            \n",
    "                                                        \n",
    "                                                    </td>\n",
    "                                                \n",
    "                                            \n",
    "                                                \n",
    "                                                    <td class=\"\" data-value=\"125\">\n",
    "                                                        \n",
    "                                                            \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                \n",
    "                                                                    125 W\n",
    "                                                                \n",
    "                                                            \n",
    "                                                        \n",
    "                                                    </td>\n",
    "                                                \n",
    "                                            \n",
    "                                        </tr>\n",
    "                                    \n",
    "                                   ```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AI Prompt**\n",
    "\n",
    "How to automate extraction ?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AI Response**\n",
    "\n",
    "I see you're looking to extract CPU data from Intel's website. Before diving into the automation, I'd like to understand a few things:\n",
    "\n",
    "1. Are you comfortable with Python programming? \n",
    "2. Have you tried web scraping before?\n",
    "3. Would you prefer using a specific library (like BeautifulSoup, Scrapy, or Selenium)?\n",
    "\n",
    "This looks like a dynamic table with sortable columns, so we'll need to consider the best approach based on your experience level and requirements. Let me know your preferences, and I'll help guide you through the process."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AI Prompt**\n",
    "\n",
    "BeautifulSoup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AI Response**\n",
    "\n",
    "Great choice! BeautifulSoup is perfect for this. Looking at the HTML you shared, we can extract the table data. Would you like me to:\n",
    "\n",
    "1. Help you write the code step by step, or\n",
    "2. Provide a complete solution?\n",
    "\n",
    "Also, do you need to:\n",
    "- Extract just this one page, or\n",
    "- Navigate through multiple pages?\n",
    "\n",
    "Let me know your preference and I'll help accordingly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AI Prompt**\n",
    "\n",
    "Extract just this one page"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AI Response**\n",
    "\n",
    "I'll help you extract data from this page. Let's break it down into steps:\n",
    "\n",
    "1. First, we'll need to set up BeautifulSoup and make the request. Would you like to start by writing the code to import the required libraries and fetch the page?\n",
    "\n",
    "The libraries we'll need are:\n",
    "- `beautifulsoup4`\n",
    "- `requests`\n",
    "\n",
    "Would you like to try writing this first part, or would you like me to show you how?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AI Prompt**\n",
    "\n",
    "give me the code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AI Response**\n",
    "\n",
    "Here's the code to get started:\n",
    "\n",
    "```python\n",
    "import requests\n",
    "from bs4 import BeautifulSoup\n",
    "\n",
    "url = \"https://www.intel.com/content/www/us/en/ark/products/series/236644/5th-gen-intel-xeon-scalable-processors.html\"\n",
    "response = requests.get(url)\n",
    "soup = BeautifulSoup(response.text, 'html.parser')\n",
    "```\n",
    "\n",
    "Try running this code. Let me know if it works, and then we'll move on to extracting the table data. \n",
    "\n",
    "(Note: If you get any errors about missing libraries, you'll need to install them first using `pip install requests beautifulsoup4`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "from bs4 import BeautifulSoup\n",
    "\n",
    "url = \"https://www.intel.com/content/www/us/en/ark/products/series/236644/5th-gen-intel-xeon-scalable-processors.html\"\n",
    "response = requests.get(url)\n",
    "soup = BeautifulSoup(response.text, \"html.parser\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AI Prompt**\n",
    "\n",
    "fine, give me the code to extract the data ?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**AI Response**\n",
    "\n",
    "Here's the code to extract the table data:\n",
    "\n",
    "```python\n",
    "# Find the table\n",
    "table = soup.find('table', id='product-table')\n",
    "\n",
    "# Extract headers\n",
    "headers = []\n",
    "for th in table.find_all('th'):\n",
    "    header_text = th.find('div', class_='header-text-space').text.strip()\n",
    "    headers.append(header_text)\n",
    "\n",
    "# Extract rows\n",
    "data = []\n",
    "for row in table.find('tbody').find_all('tr'):\n",
    "    row_data = {}\n",
    "    cells = row.find_all('td')\n",
    "    for i, cell in enumerate(cells):\n",
    "        row_data[headers[i]] = cell.text.strip()\n",
    "    data.append(row_data)\n",
    "\n",
    "# Convert to pandas DataFrame for easy viewing/export\n",
    "import pandas as pd\n",
    "df = pd.DataFrame(data)\n",
    "print(df)\n",
    "```\n",
    "\n",
    "Would you like me to explain how this code works, or would you prefer to see how to modify it for specific needs?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                              Product Name Launch Date  ...    Cache    TDP\n",
      "0      Intel® Xeon® Bronze 3508U Processor       Q4'23  ...  22.5 MB  125 W\n",
      "1        Intel® Xeon® Gold 5512U Processor       Q4'23  ...  52.5 MB  185 W\n",
      "2        Intel® Xeon® Gold 5515+ Processor       Q4'23  ...  22.5 MB  165 W\n",
      "3        Intel® Xeon® Gold 5520+ Processor       Q4'23  ...  52.5 MB  205 W\n",
      "4        Intel® Xeon® Gold 6526Y Processor       Q4'23  ...  37.5 MB  195 W\n",
      "5         Intel® Xeon® Gold 6530 Processor       Q4'23  ...   160 MB  270 W\n",
      "6         Intel® Xeon® Gold 6534 Processor       Q4'23  ...  22.5 MB  195 W\n",
      "7        Intel® Xeon® Gold 6538N Processor       Q4'23  ...    60 MB  205 W\n",
      "8       Intel® Xeon® Gold 6538Y+ Processor       Q4'23  ...    60 MB  225 W\n",
      "9        Intel® Xeon® Gold 6542Y Processor       Q4'23  ...    60 MB  250 W\n",
      "10       Intel® Xeon® Gold 6544Y Processor       Q4'23  ...    45 MB  270 W\n",
      "11       Intel® Xeon® Gold 6548N Processor       Q4'23  ...    60 MB  250 W\n",
      "12      Intel® Xeon® Gold 6548Y+ Processor       Q4'23  ...    60 MB  250 W\n",
      "13       Intel® Xeon® Gold 6554S Processor       Q4'23  ...   180 MB  270 W\n",
      "14       Intel® Xeon® Gold 6558Q Processor       Q4'23  ...    60 MB  350 W\n",
      "15    Intel® Xeon® Platinum 8558 Processor       Q4'23  ...   260 MB  330 W\n",
      "16   Intel® Xeon® Platinum 8558P Processor       Q4'23  ...   260 MB  350 W\n",
      "17   Intel® Xeon® Platinum 8558U Processor       Q4'23  ...   260 MB  300 W\n",
      "18  Intel® Xeon® Platinum 8562Y+ Processor       Q4'23  ...    60 MB  300 W\n",
      "19  Intel® Xeon® Platinum 8568Y+ Processor       Q4'23  ...   300 MB  350 W\n",
      "20    Intel® Xeon® Platinum 8570 Processor       Q4'23  ...   300 MB  350 W\n",
      "21   Intel® Xeon® Platinum 8571N Processor       Q4'23  ...   300 MB  300 W\n",
      "22    Intel® Xeon® Platinum 8580 Processor       Q4'23  ...   300 MB  350 W\n",
      "23   Intel® Xeon® Platinum 8581V Processor       Q4'23  ...   300 MB  270 W\n",
      "24   Intel® Xeon® Platinum 8592+ Processor       Q4'23  ...   320 MB  350 W\n",
      "25   Intel® Xeon® Platinum 8592V Processor       Q4'23  ...   320 MB  330 W\n",
      "26   Intel® Xeon® Platinum 8593Q Processor       Q4'23  ...   320 MB  385 W\n",
      "27     Intel® Xeon® Silver 4509Y Processor       Q4'23  ...  22.5 MB  125 W\n",
      "28      Intel® Xeon® Silver 4510 Processor       Q4'23  ...    30 MB  150 W\n",
      "29     Intel® Xeon® Silver 4510T Processor       Q4'23  ...    30 MB  115 W\n",
      "30     Intel® Xeon® Silver 4514Y Processor       Q4'23  ...    30 MB  150 W\n",
      "31    Intel® Xeon® Silver 4516Y+ Processor       Q4'23  ...    45 MB  185 W\n",
      "\n",
      "[32 rows x 7 columns]\n"
     ]
    }
   ],
   "source": [
    "# Find the table\n",
    "table = soup.find(\"table\", id=\"product-table\")\n",
    "\n",
    "# Extract headers\n",
    "headers = []\n",
    "for th in table.find_all(\"th\"):\n",
    "    header_text = th.find(\"div\", class_=\"header-text-space\").text.strip()\n",
    "    headers.append(header_text)\n",
    "\n",
    "# Extract rows\n",
    "data = []\n",
    "for row in table.find(\"tbody\").find_all(\"tr\"):\n",
    "    row_data = {}\n",
    "    cells = row.find_all(\"td\")\n",
    "    for i, cell in enumerate(cells):\n",
    "        row_data[headers[i]] = cell.text.strip()\n",
    "    data.append(row_data)\n",
    "\n",
    "# Convert to pandas DataFrame for easy viewing/export\n",
    "import pandas as pd\n",
    "\n",
    "df = pd.DataFrame(data)\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     125 W\n",
       "1     185 W\n",
       "2     165 W\n",
       "3     205 W\n",
       "4     195 W\n",
       "5     270 W\n",
       "6     195 W\n",
       "7     205 W\n",
       "8     225 W\n",
       "9     250 W\n",
       "10    270 W\n",
       "11    250 W\n",
       "12    250 W\n",
       "13    270 W\n",
       "14    350 W\n",
       "15    330 W\n",
       "16    350 W\n",
       "17    300 W\n",
       "18    300 W\n",
       "19    350 W\n",
       "20    350 W\n",
       "21    300 W\n",
       "22    350 W\n",
       "23    270 W\n",
       "24    350 W\n",
       "25    330 W\n",
       "26    385 W\n",
       "27    125 W\n",
       "28    150 W\n",
       "29    115 W\n",
       "30    150 W\n",
       "31    185 W\n",
       "Name: TDP, dtype: object"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"TDP\"].replace(\"W\", \"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## AMD"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average TDP per core from AMD dataset: 4.73 W\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "# Path to your manually exported AMD CPU dataset.\n",
    "# (Adjust the file path as needed.)\n",
    "amd_csv_path = \"./AMD_CPU_desktop_laptop.csv\"\n",
    "\n",
    "try:\n",
    "    amd_df = pd.read_csv(amd_csv_path)\n",
    "    amd_df = amd_df[amd_df[\"Launch Date\"].str.contains(\"2024|2025\", na=False)]\n",
    "    amd_df = amd_df[amd_df[\"Form Factor\"].str.contains(\"Desktops\", na=False)]\n",
    "\n",
    "    # Convert columns to numeric, forcing errors to NaN\n",
    "    # amd_df['Default TDP'] = amd_df['Default TDP'].str.replace('W', '').astype(float)\n",
    "    amd_df[\"TDP\"] = pd.to_numeric(\n",
    "        amd_df[\"Default TDP\"].str.replace(\"W\", \"\"), errors=\"coerce\"\n",
    "    )\n",
    "    amd_df[\"# of Threads\"] = pd.to_numeric(amd_df[\"# of Threads\"], errors=\"coerce\")\n",
    "\n",
    "    # It is assumed the CSV contains columns named 'TDP' (in Watts) and 'Total Cores'\n",
    "    # Adjust the column names if they differ.\n",
    "    amd_df[\"TDP_per_core\"] = amd_df[\"TDP\"] / amd_df[\"# of Threads\"]\n",
    "\n",
    "    average_tdp_per_core = amd_df[\"TDP_per_core\"].mean()\n",
    "    print(\n",
    "        \"Average TDP per core from AMD dataset: {:.2f} W\".format(average_tdp_per_core)\n",
    "    )\n",
    "except Exception as e:\n",
    "    print(\"Error loading or processing AMD dataset:\", e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average TDP per core from AMD dataset: 3.61 W\n"
     ]
    }
   ],
   "source": [
    "# https://www.amd.com/en/products/specifications/server-processor.html\n",
    "import pandas as pd\n",
    "\n",
    "# Path to your manually exported AMD CPU dataset.\n",
    "# (Adjust the file path as needed.)\n",
    "amd_csv_path = \"./AMD_Server_Processor_Specifications.csv\"\n",
    "\n",
    "try:\n",
    "    amd_df = pd.read_csv(amd_csv_path)\n",
    "    amd_df = amd_df[amd_df[\"Launch Date\"].str.contains(\"2024|2025\", na=False)]\n",
    "\n",
    "    # Convert columns to numeric, forcing errors to NaN\n",
    "    # amd_df['Default TDP'] = amd_df['Default TDP'].str.replace('W', '').astype(float)\n",
    "    amd_df[\"TDP\"] = pd.to_numeric(\n",
    "        amd_df[\"Default TDP\"].str.replace(\"W\", \"\"), errors=\"coerce\"\n",
    "    )\n",
    "    amd_df[\"# of Threads\"] = pd.to_numeric(amd_df[\"# of Threads\"], errors=\"coerce\")\n",
    "\n",
    "    # It is assumed the CSV contains columns named 'TDP' (in Watts) and 'Total Cores'\n",
    "    # Adjust the column names if they differ.\n",
    "    amd_df[\"TDP_per_core\"] = amd_df[\"TDP\"] / amd_df[\"# of Threads\"]\n",
    "\n",
    "    average_tdp_per_core = amd_df[\"TDP_per_core\"].mean()\n",
    "    print(\n",
    "        \"Average TDP per core from AMD dataset: {:.2f} W\".format(average_tdp_per_core)\n",
    "    )\n",
    "except Exception as e:\n",
    "    print(\"Error loading or processing AMD dataset:\", e)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Name', 'Family', 'Series', 'Form Factor', '# of CPU Cores',\n",
       "       '# of Threads', 'Max. Boost Clock', 'Base Clock', 'L2 Cache',\n",
       "       'L3 Cache', 'Default TDP', 'L1 Cache', 'AMD Configurable TDP (cTDP)',\n",
       "       'Processor Technology for CPU Cores', 'Unlocked for Overclocking',\n",
       "       'CPU Socket', 'Thermal Solution (PIB)', 'Recommended Cooler',\n",
       "       'Thermal Solution (MPK)', 'Max. Operating Temperature (Tjmax)',\n",
       "       'Launch Date', '*OS Support', 'PCI Express® Version',\n",
       "       'System Memory Type', 'Memory Channels', 'System Memory Specification',\n",
       "       'Graphics Model', 'Graphics Core Count', 'Graphics Frequency',\n",
       "       'AMD Ryzen™ AI', 'Product ID Boxed', 'Product ID Tray',\n",
       "       'Product ID MPK', 'Supported Technologies', 'TDP_per_core'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "amd_df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>TDP</th>\n",
       "      <th># of Threads</th>\n",
       "      <th>TDP_per_core</th>\n",
       "      <th>Launch Date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>AMD Ryzen™ 9 9950X</td>\n",
       "      <td>170</td>\n",
       "      <td>32.0</td>\n",
       "      <td>5.312500</td>\n",
       "      <td>08/15/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>AMD Ryzen™ 9 9900X</td>\n",
       "      <td>120</td>\n",
       "      <td>24.0</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>08/15/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>AMD Ryzen™ 7 9800X3D</td>\n",
       "      <td>120</td>\n",
       "      <td>16.0</td>\n",
       "      <td>7.500000</td>\n",
       "      <td>11/07/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>AMD Ryzen™ 7 9700X</td>\n",
       "      <td>65</td>\n",
       "      <td>16.0</td>\n",
       "      <td>4.062500</td>\n",
       "      <td>08/08/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>AMD Ryzen™ 5 9600X</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>08/08/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>AMD Ryzen™ 7 PRO 8845HS</td>\n",
       "      <td>45</td>\n",
       "      <td>16.0</td>\n",
       "      <td>2.812500</td>\n",
       "      <td>04/16/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>AMD Ryzen™ 7 PRO 8700GE</td>\n",
       "      <td>35</td>\n",
       "      <td>16.0</td>\n",
       "      <td>2.187500</td>\n",
       "      <td>4/16/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>AMD Ryzen™ 7 PRO 8700G</td>\n",
       "      <td>65</td>\n",
       "      <td>16.0</td>\n",
       "      <td>4.062500</td>\n",
       "      <td>4/16/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>AMD Ryzen™ 5 PRO 8600GE</td>\n",
       "      <td>35</td>\n",
       "      <td>12.0</td>\n",
       "      <td>2.916667</td>\n",
       "      <td>4/16/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>AMD Ryzen™ 5 PRO 8600G</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>4/16/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54</th>\n",
       "      <td>AMD Ryzen™ 5 PRO 8500GE</td>\n",
       "      <td>35</td>\n",
       "      <td>12.0</td>\n",
       "      <td>2.916667</td>\n",
       "      <td>4/16/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55</th>\n",
       "      <td>AMD Ryzen™ 5 PRO 8500G</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>4/16/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>AMD Ryzen™ 3 PRO 8300GE</td>\n",
       "      <td>35</td>\n",
       "      <td>8.0</td>\n",
       "      <td>4.375000</td>\n",
       "      <td>4/16/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>AMD Ryzen™ 3 PRO 8300G</td>\n",
       "      <td>65</td>\n",
       "      <td>8.0</td>\n",
       "      <td>8.125000</td>\n",
       "      <td>4/16/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62</th>\n",
       "      <td>AMD Ryzen™ 7 8700G</td>\n",
       "      <td>65</td>\n",
       "      <td>16.0</td>\n",
       "      <td>4.062500</td>\n",
       "      <td>1/31/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>63</th>\n",
       "      <td>AMD Ryzen™ 7 8700F</td>\n",
       "      <td>65</td>\n",
       "      <td>16.0</td>\n",
       "      <td>4.062500</td>\n",
       "      <td>04/01/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67</th>\n",
       "      <td>AMD Ryzen™ 5 8600G</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>1/31/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>69</th>\n",
       "      <td>AMD Ryzen™ 5 8500GE</td>\n",
       "      <td>35</td>\n",
       "      <td>12.0</td>\n",
       "      <td>2.916667</td>\n",
       "      <td>4/16/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>AMD Ryzen™ 5 8500G</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>1/31/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>AMD Ryzen™ 3 8300GE</td>\n",
       "      <td>35</td>\n",
       "      <td>8.0</td>\n",
       "      <td>4.375000</td>\n",
       "      <td>4/16/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>AMD Ryzen™ 5 8400F</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>04/01/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>AMD Ryzen™ 3 8300G</td>\n",
       "      <td>65</td>\n",
       "      <td>8.0</td>\n",
       "      <td>8.125000</td>\n",
       "      <td>1/31/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>105</th>\n",
       "      <td>AMD Ryzen™ 9 7940HX</td>\n",
       "      <td>55</td>\n",
       "      <td>32.0</td>\n",
       "      <td>1.718750</td>\n",
       "      <td>1/17/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>111</th>\n",
       "      <td>AMD Ryzen™ 7 7840HX</td>\n",
       "      <td>55</td>\n",
       "      <td>24.0</td>\n",
       "      <td>2.291667</td>\n",
       "      <td>1/17/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>AMD Ryzen™ 5 7600X3D</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>9/5/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>138</th>\n",
       "      <td>AMD Ryzen™ 5 7400F</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>1/9/2025</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>178</th>\n",
       "      <td>AMD Ryzen™ 7 PRO 5755GE</td>\n",
       "      <td>35</td>\n",
       "      <td>16.0</td>\n",
       "      <td>2.187500</td>\n",
       "      <td>9/5/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>179</th>\n",
       "      <td>AMD Ryzen™ 7 PRO 5755G</td>\n",
       "      <td>65</td>\n",
       "      <td>16.0</td>\n",
       "      <td>4.062500</td>\n",
       "      <td>9/5/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183</th>\n",
       "      <td>AMD Ryzen™ 5 PRO 5655GE</td>\n",
       "      <td>35</td>\n",
       "      <td>12.0</td>\n",
       "      <td>2.916667</td>\n",
       "      <td>5/7/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>184</th>\n",
       "      <td>AMD Ryzen™ 5 PRO 5655G</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>5/7/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>191</th>\n",
       "      <td>AMD Ryzen™ 3 PRO 5355GE</td>\n",
       "      <td>35</td>\n",
       "      <td>8.0</td>\n",
       "      <td>4.375000</td>\n",
       "      <td>9/5/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>192</th>\n",
       "      <td>AMD Ryzen™ 3 PRO 5355G</td>\n",
       "      <td>65</td>\n",
       "      <td>8.0</td>\n",
       "      <td>8.125000</td>\n",
       "      <td>9/5/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>198</th>\n",
       "      <td>AMD Ryzen™ 9 5900XT</td>\n",
       "      <td>105</td>\n",
       "      <td>32.0</td>\n",
       "      <td>3.281250</td>\n",
       "      <td>07/31/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>206</th>\n",
       "      <td>AMD Ryzen™ 7 5800XT</td>\n",
       "      <td>105</td>\n",
       "      <td>16.0</td>\n",
       "      <td>6.562500</td>\n",
       "      <td>07/31/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>214</th>\n",
       "      <td>AMD Ryzen™ 7 5700X3D</td>\n",
       "      <td>105</td>\n",
       "      <td>16.0</td>\n",
       "      <td>6.562500</td>\n",
       "      <td>01/08/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>219</th>\n",
       "      <td>AMD Ryzen™ 7 5700</td>\n",
       "      <td>65</td>\n",
       "      <td>16.0</td>\n",
       "      <td>4.062500</td>\n",
       "      <td>01/31/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>229</th>\n",
       "      <td>AMD Ryzen™ 5 5600XT</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>10/31/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>230</th>\n",
       "      <td>AMD Ryzen™ 5 5600T</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>10/31/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>231</th>\n",
       "      <td>AMD Ryzen™ 5 5600GT</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>01/08/2024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>238</th>\n",
       "      <td>AMD Ryzen™ 5 5500GT</td>\n",
       "      <td>65</td>\n",
       "      <td>12.0</td>\n",
       "      <td>5.416667</td>\n",
       "      <td>01/08/2024</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        Name  TDP  # of Threads  TDP_per_core Launch Date\n",
       "34        AMD Ryzen™ 9 9950X  170          32.0      5.312500  08/15/2024\n",
       "36        AMD Ryzen™ 9 9900X  120          24.0      5.000000  08/15/2024\n",
       "38      AMD Ryzen™ 7 9800X3D  120          16.0      7.500000  11/07/2024\n",
       "39        AMD Ryzen™ 7 9700X   65          16.0      4.062500  08/08/2024\n",
       "40        AMD Ryzen™ 5 9600X   65          12.0      5.416667  08/08/2024\n",
       "44   AMD Ryzen™ 7 PRO 8845HS   45          16.0      2.812500  04/16/2024\n",
       "46   AMD Ryzen™ 7 PRO 8700GE   35          16.0      2.187500   4/16/2024\n",
       "47    AMD Ryzen™ 7 PRO 8700G   65          16.0      4.062500   4/16/2024\n",
       "51   AMD Ryzen™ 5 PRO 8600GE   35          12.0      2.916667   4/16/2024\n",
       "52    AMD Ryzen™ 5 PRO 8600G   65          12.0      5.416667   4/16/2024\n",
       "54   AMD Ryzen™ 5 PRO 8500GE   35          12.0      2.916667   4/16/2024\n",
       "55    AMD Ryzen™ 5 PRO 8500G   65          12.0      5.416667   4/16/2024\n",
       "57   AMD Ryzen™ 3 PRO 8300GE   35           8.0      4.375000   4/16/2024\n",
       "58    AMD Ryzen™ 3 PRO 8300G   65           8.0      8.125000   4/16/2024\n",
       "62        AMD Ryzen™ 7 8700G   65          16.0      4.062500   1/31/2024\n",
       "63        AMD Ryzen™ 7 8700F   65          16.0      4.062500  04/01/2024\n",
       "67        AMD Ryzen™ 5 8600G   65          12.0      5.416667   1/31/2024\n",
       "69       AMD Ryzen™ 5 8500GE   35          12.0      2.916667   4/16/2024\n",
       "70        AMD Ryzen™ 5 8500G   65          12.0      5.416667   1/31/2024\n",
       "72       AMD Ryzen™ 3 8300GE   35           8.0      4.375000   4/16/2024\n",
       "73        AMD Ryzen™ 5 8400F   65          12.0      5.416667  04/01/2024\n",
       "74        AMD Ryzen™ 3 8300G   65           8.0      8.125000   1/31/2024\n",
       "105      AMD Ryzen™ 9 7940HX   55          32.0      1.718750   1/17/2024\n",
       "111      AMD Ryzen™ 7 7840HX   55          24.0      2.291667   1/17/2024\n",
       "124     AMD Ryzen™ 5 7600X3D   65          12.0      5.416667    9/5/2024\n",
       "138       AMD Ryzen™ 5 7400F   65          12.0      5.416667    1/9/2025\n",
       "178  AMD Ryzen™ 7 PRO 5755GE   35          16.0      2.187500    9/5/2024\n",
       "179   AMD Ryzen™ 7 PRO 5755G   65          16.0      4.062500    9/5/2024\n",
       "183  AMD Ryzen™ 5 PRO 5655GE   35          12.0      2.916667    5/7/2024\n",
       "184   AMD Ryzen™ 5 PRO 5655G   65          12.0      5.416667    5/7/2024\n",
       "191  AMD Ryzen™ 3 PRO 5355GE   35           8.0      4.375000    9/5/2024\n",
       "192   AMD Ryzen™ 3 PRO 5355G   65           8.0      8.125000    9/5/2024\n",
       "198      AMD Ryzen™ 9 5900XT  105          32.0      3.281250  07/31/2024\n",
       "206      AMD Ryzen™ 7 5800XT  105          16.0      6.562500  07/31/2024\n",
       "214     AMD Ryzen™ 7 5700X3D  105          16.0      6.562500  01/08/2024\n",
       "219        AMD Ryzen™ 7 5700   65          16.0      4.062500  01/31/2024\n",
       "229      AMD Ryzen™ 5 5600XT   65          12.0      5.416667  10/31/2024\n",
       "230       AMD Ryzen™ 5 5600T   65          12.0      5.416667  10/31/2024\n",
       "231      AMD Ryzen™ 5 5600GT   65          12.0      5.416667  01/08/2024\n",
       "238      AMD Ryzen™ 5 5500GT   65          12.0      5.416667  01/08/2024"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "amd_df[[\"Name\", \"TDP\", \"# of Threads\", \"TDP_per_core\", \"Launch Date\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Update the static CodeCarbon database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>TDP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1075T</td>\n",
       "      <td>95</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>255e</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3260 HE</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3280</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3320 EE</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3923</th>\n",
       "      <td>TL-52</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3924</th>\n",
       "      <td>TL-60</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3925</th>\n",
       "      <td>TL-64</td>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3926</th>\n",
       "      <td>X1150</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3927</th>\n",
       "      <td>X940</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3928 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         Name TDP\n",
       "0       1075T  95\n",
       "1        255e  45\n",
       "2     3260 HE  45\n",
       "3        3280  65\n",
       "4     3320 EE  25\n",
       "...       ...  ..\n",
       "3923    TL-52  31\n",
       "3924    TL-60  31\n",
       "3925    TL-64  35\n",
       "3926    X1150  17\n",
       "3927     X940  45\n",
       "\n",
       "[3928 rows x 2 columns]"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"cpu_power.csv\")\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>TDP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AMD EPYC 9965</td>\n",
       "      <td>500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AMD EPYC 9845</td>\n",
       "      <td>390.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AMD EPYC 9825</td>\n",
       "      <td>390.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Name    TDP\n",
       "0  AMD EPYC 9965  500.0\n",
       "1  AMD EPYC 9845  390.0\n",
       "2  AMD EPYC 9825  390.0"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "amd_csv_path = \"./AMD_Server_Processor_Specifications.csv\"\n",
    "amd_df = pd.read_csv(amd_csv_path)\n",
    "amd_df[\"TDP\"] = pd.to_numeric(\n",
    "    amd_df[\"Default TDP\"].str.replace(\"W\", \"\"), errors=\"coerce\"\n",
    ")\n",
    "amd_df[\"Name\"] = amd_df[\"Name\"].str.replace(\"™\", \"\")\n",
    "amd_server = amd_df[[\"Name\", \"TDP\"]]\n",
    "amd_server = amd_server.dropna(subset=[\"TDP\"])\n",
    "amd_server.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_52381/4222190993.py:6: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  new_cpus['TDP'] = new_cpus['TDP_AMD']\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>TDP</th>\n",
       "      <th>TDP_AMD</th>\n",
       "      <th>_merge</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>505</th>\n",
       "      <td>AMD EPYC 4124P</td>\n",
       "      <td>65</td>\n",
       "      <td>65</td>\n",
       "      <td>right_only</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>506</th>\n",
       "      <td>AMD EPYC 4244P</td>\n",
       "      <td>65</td>\n",
       "      <td>65</td>\n",
       "      <td>right_only</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>507</th>\n",
       "      <td>AMD EPYC 4344P</td>\n",
       "      <td>65</td>\n",
       "      <td>65</td>\n",
       "      <td>right_only</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>508</th>\n",
       "      <td>AMD EPYC 4364P</td>\n",
       "      <td>105</td>\n",
       "      <td>105</td>\n",
       "      <td>right_only</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>509</th>\n",
       "      <td>AMD EPYC 4464P</td>\n",
       "      <td>65</td>\n",
       "      <td>65</td>\n",
       "      <td>right_only</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>752</th>\n",
       "      <td>AMD Opteron 6328</td>\n",
       "      <td>115</td>\n",
       "      <td>115</td>\n",
       "      <td>right_only</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>753</th>\n",
       "      <td>AMD Opteron 6338P</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "      <td>right_only</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>754</th>\n",
       "      <td>AMD Opteron 6344</td>\n",
       "      <td>115</td>\n",
       "      <td>115</td>\n",
       "      <td>right_only</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>757</th>\n",
       "      <td>AMD Opteron 6370P</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "      <td>right_only</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>807</th>\n",
       "      <td>AMD Opteron X2170 APU</td>\n",
       "      <td>25</td>\n",
       "      <td>25</td>\n",
       "      <td>right_only</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>113 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                      Name  TDP TDP_AMD      _merge\n",
       "505         AMD EPYC 4124P   65      65  right_only\n",
       "506         AMD EPYC 4244P   65      65  right_only\n",
       "507         AMD EPYC 4344P   65      65  right_only\n",
       "508         AMD EPYC 4364P  105     105  right_only\n",
       "509         AMD EPYC 4464P   65      65  right_only\n",
       "..                     ...  ...     ...         ...\n",
       "752       AMD Opteron 6328  115     115  right_only\n",
       "753      AMD Opteron 6338P   99      99  right_only\n",
       "754       AMD Opteron 6344  115     115  right_only\n",
       "757      AMD Opteron 6370P   99      99  right_only\n",
       "807  AMD Opteron X2170 APU   25      25  right_only\n",
       "\n",
       "[113 rows x 4 columns]"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Merge df with amd_server, avoiding duplicates\n",
    "amd_server[\"TDP\"] = amd_server[\"TDP\"].astype(int).astype(str)\n",
    "merged_df = df.merge(\n",
    "    amd_server, on=\"Name\", how=\"outer\", suffixes=(\"\", \"_AMD\"), indicator=True\n",
    ")\n",
    "# Filter for new entries that are only in amd_server\n",
    "new_cpus = merged_df[merged_df[\"_merge\"] == \"right_only\"]\n",
    "new_cpus[\"TDP\"] = new_cpus[\"TDP_AMD\"]\n",
    "new_cpus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>TDP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>505</th>\n",
       "      <td>AMD EPYC 4124P</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>506</th>\n",
       "      <td>AMD EPYC 4244P</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>507</th>\n",
       "      <td>AMD EPYC 4344P</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>508</th>\n",
       "      <td>AMD EPYC 4364P</td>\n",
       "      <td>105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>509</th>\n",
       "      <td>AMD EPYC 4464P</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>752</th>\n",
       "      <td>AMD Opteron 6328</td>\n",
       "      <td>115</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>753</th>\n",
       "      <td>AMD Opteron 6338P</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>754</th>\n",
       "      <td>AMD Opteron 6344</td>\n",
       "      <td>115</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>757</th>\n",
       "      <td>AMD Opteron 6370P</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>807</th>\n",
       "      <td>AMD Opteron X2170 APU</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>113 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                      Name  TDP\n",
       "505         AMD EPYC 4124P   65\n",
       "506         AMD EPYC 4244P   65\n",
       "507         AMD EPYC 4344P   65\n",
       "508         AMD EPYC 4364P  105\n",
       "509         AMD EPYC 4464P   65\n",
       "..                     ...  ...\n",
       "752       AMD Opteron 6328  115\n",
       "753      AMD Opteron 6338P   99\n",
       "754       AMD Opteron 6344  115\n",
       "757      AMD Opteron 6370P   99\n",
       "807  AMD Opteron X2170 APU   25\n",
       "\n",
       "[113 rows x 2 columns]"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# merged_df.query('Name.str.contains(\"EPYC\")')\n",
    "new_cpus_to_add = new_cpus.drop(columns=[\"_merge\"]).loc[:, df.columns]\n",
    "new_cpus_to_add"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>TDP</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3928</th>\n",
       "      <td>AMD EPYC 4124P</td>\n",
       "      <td>65</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                Name TDP\n",
       "3928  AMD EPYC 4124P  65"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Option 2: Append the new CPUs to the original df\n",
    "df = pd.concat([df, new_cpus_to_add], ignore_index=True)\n",
    "df.sort_values(\"Name\", ascending=True, inplace=True)\n",
    "df.query('Name.str.contains(\"AMD EPYC 4124P\")')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(\"cpu_power.csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Remove with..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "# Load the existing dataset\n",
    "df = pd.read_csv(\"../cpu_power.csv\")\n",
    "# Replace with re.sub(r\" with.*\", \"\", name)\n",
    "def clean_cpu_name(name):\n",
    "    import re\n",
    "    # Remove \"with\" and everything after it\n",
    "    name = re.sub(r\" with.*\", \"\", name)\n",
    "    # Remove \"™\" symbol\n",
    "    name = name.replace(\"™\", \"\")\n",
    "    return name.strip()\n",
    "df[\"Name\"] = df[\"Name\"].apply(clean_cpu_name)\n",
    "# Save the cleaned dataset\n",
    "df.to_csv(\"../cpu_power.csv\", index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "3.10.5",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
